Accelerated Variational Infinite Mixture Models

Infinite mixture models, such as the Dirichlet process mixture, are promising candidates for clustering applications where the number of clusters is unknown a priori. Due to computational considerations these models are unfortunately unsuitable for large scale data-mining applications. We propose a class of deterministic accelerated infinite mixture models that can routinely handle millions of data-cases. The speedup is achieved by incorporating kd-trees into a variational Bayesian algorithm for infinite mixture models in the stick breaking representation. Besides kd-trees, this algorithm is also different from Blei and Jordan (2005) in the way we handle truncation: we only assume that the variational distributions are fixed at their priors after a certain truncation level. Experiments show that speedups relative to the standard variational algorithm can be significant.